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The  analysis  of  aircraft  data  recorded  during  flight  has  played  a  crucial 
role  in  determining  the  causes  of  crashes.  Recently,  however,  some  U.S. 
airlines  have  begun  to  analyze  flight  data  from  uneventful  airline  flights  to 
identify  potential  problems  and  correct  them  before  they  lead  to  accidents. 
In  your  letter  of  December  2, 1996,  you  asked  us  to  examine  efforts  by  the 
Federal  Aviation  Administration  (faa)  and  U.S.  airlines  to  implement 
Flight  Operational  Quality  Assurance  (foqa)  programs.  The  objective  of  a 
FOQA  program  is  to  use  flight  data  to  detect  technical  flaws,  unsafe 
practices,  or  conditions  outside  of  desired  operating  procedures  early 
enough  to  allow  timely  intervention  to  avert  accidents  or  incidents.  These 
programs  are  voluntary  efforts  by  airlines  that  involve  equipping  aircraft 
vdth  specialized  devices  to  continuously  record  up  to  hundreds  of 
different  flight  data  parameters  from  aircraft  systems  and  sensors, 
analyzing  the  data,  identifying  trends,  and  taking  action  to  correct 
potential  problems.  The  analysis  of  flight  data  allows  airlines  to 
reconstruct  entire  flights  on  the  basis  of  the  values  over  time  of  flight  data 
parameters,  such  as  heading,  altitude,  throttle  settings,  ground  speed,  and 
many  others.  Currently,  about  33  foreign  airlines  and  4  U.S. 
airlines — United,  US  Airways,  Continental,  and  Alaska — ^have  implemented 
FOQA  or  FOQA-type  programs. 

You  requested  that  we  determine  (1)  how  foqa  programs  will  enhance 
aviation  safety,  (2)  the  costs  and  benefits  of  such  programs,  and  (3)  the 
factors  that  could  impede  their  full  implementation  and  actions  that  could 
be  taken  to  overcome  any  impediments. 


Results  in  Brief 


The  early  experience  of  domestic  airlines  with  established  Flight 
Operational  Quality  Assurance  programs,  as  well  as  the  testimony  of 
foreign  airlines  with  extensive  experience  in  this  area,  attests  to  the 
potential  of  such  programs  to  enhance  aviation  safety  by  identifying 
possible  safety  problems  that  could  lead  to  accidents.  Airlines  have  used 
Flight  Operational  Qualify  Assurance  programs  to  identify  potential 
problems  that  were  previously  unknovm  or  only  suspected.  Where 
potential  problems  were  already  known,  airUnes  have  used  these 
programs  to  confirm  and  quantify  the  extent  of  the  problems.  And  most 
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important,  on  the  basis  of  analyses  of  flight  data,  airlines  have  taken 
actions  to  correct  problems  and  enhance  aviation  safety. 

The  costs  associated  with  implementing  a  Flight  Operational  Quality 
Assurance  program  depend  upon  a  large  number  of  factors,  including  the 
technology  used  to  capture  flight  data,  the  number  and  types  of  aircraft  to 
be  equipped  with  this  technology,  and  personnel  costs.  Although  the 
program  is  primarily  viewed  as  a  safety  program,  U.S.  and  foreign  airlines 
have  reported  financial  benefits  as  well.  With  additional  data  on  aircraft 
systems  and  engine  conditions,  airlines  are  better  able  to  achieve  optimum 
fuel  consumption  and  avoid  unneeded  engine  maintenance.  Although  more 
difficult  to  quantify,  enhanced  safety  should  result  in  lower  costs  over  time 
as  a  result  of  accidents  avoided  and  lower  insurance  premiums,  faa’s 
preliminary  estimates  place  the  annual  cost  of  a  program  with  50  aircraft 
at  approximately  $760,000.  Savings  firom  reduced  expenditiues  for  fuel, 
engine  maintenance,  and  accident  costs  for  a  50-aircraft  program  are 
estimated  at  $1.65  million  per  year,  paa’s  estimates  suggest  a  net  savings 
firom  50  aircraft  of  $892,000  per  year. 

The  primary  factor  impeding  the  implementation  of  Flight  Operational 
Quality  Assurance  programs  among  the  major  domestic  carriers  is  the 
resolution  of  data  protection  issues.  Airline  managers  and  pilots  raise 
three  significant  data  protection  concerns:  (1)  use  of  the  data  for 
enforcement/disciplinary  purposes;  (2)  disclosure  to  the  media  and  the 
public  under  the  provisions  of  the  Freedom  of  Information  Act;  and 
(3)  disclosure  through  the  civil  litigation  discovery  process,  faa  has  taken 
a  number  of  actions  that  may  resolve  these  issues,  although  it  is  not  clear 
whether  the  aviation  community  will  be  satisfied  with  faa’s  actions.  Rrst, 
FAA  has  begun  work  on  a  rulemaking  procedure  to  establish  what 
protections  from  enforcement  actions,  if  any,  will  apply  to  information 
submitted  to  faa  under  a  Flight  Operational  Quality  Assurance  program. 
Second,  on  October  9, 1996,  the  Congress  enacted  legislation  and  faa  has 
begun  work  on  a  rulemaking  procedure  that  would  prohibit  the 
Administrator  from  disclosing  voluntarily  submitted  safety  information 
imder  certain  circumstances.  These  actions  may  ameliorate  concerns 
about  the  Freedom  of  Information  Act.  And  third,  airlines  currently  seek  to 
protect  voluntarily  collected  safety  information  firom  disclosure  in  civil 
litigation  on  a  case-by-case  basis. 


Background 


Modem  commercial  aircraft  contain  sophisticated  electronic  systems  that 
gather,  process,  and  manage  digital  data  on  many  aspects  of  flight.  These 
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data  originate  from  various  systems  and  sensors  throughout  the  aircraft. 
The  data  range  from  pilot  operations  to  the  outputs  of  sensors  and 
systems.  Some  of  these  data  are  continuously  recorded  by  the  aircraft’s 
digital  flight  data  recorder  to  help  investigators  understand  what  happened 
if  the  aircraft  is  involved  in  an  accident  or  a  serious  incident.^  Designed  to 
survive  crashes,  flight  data  recorders  typically  retain  the  data  recorded 
during  the  last  25  hours  of  flight. 

Rather  than  analyzing  flight  data  only  after  an  incident  or  accident,  some 
airlines  routinely  analyze  the  flight  data  from  regular  flights.  Their  aim  is 
to  identify  problems  that  occur  in  routine  operations  and  to  correct  these 
problems  before  they  become  accidents  or  incidents.  In  its  1992  study  for 
FAA,^  the  Flight  Safety  Foundation  coined  the  term  “Flight  Operational 
Quality  Assurance”  to  describe  this  function.  The  Foundation  defined  foqa 
as  “a  program  for  obtaining  and  analyzing  data  recorded  in  flight  to 
improve  flight  crew  performance,  air  carrier  training  programs  and 
operating  procedures,  air  traffic  control  procedures,  airport  maintenance 
and  design,  and  aircraft  operations  and  design.” 

FOQA  has  its  origin  in  the  use  of  flight  data  recorders®  as  mandated  by  the 
Civil  Aeronautics  Administration  in  1968.  Although  the  first  flight  data 
recorders  captured  only  six  parameters,^  they  were  a  valuable  tool  for 
reconstructing  what  had  occurred  preceding  a  crash.  In  addition  to 
recording  data  to  assist  in  crash  investigations,  some  airlines  began  to 
monitor  data  recorded  on  routine  flights.  Initially,  the  monitoring  systems 
captured  airworthiness  data,  but  over  time  they  have  expanded  to  include 
operational  data,  foqa  programs  were  first  established  in  Europe  and  Asia, 
and  only  within  the  past  few  years  have  some  U.S.  airlines  begun  adopting 
such  a  system  on  a  trial  basis.  At  present,  about  33  foreign  airlines  and  4 
U.S.  airlines — United,  US  Airways,  Continental,  and  Alaska — ^have 
implemented  foqa  or  FOQA-fype  programs.  (See  app.  I  for  more  detailed 
background  information  on  foqa  and  U.S.  airlines’  experience  with  these 
programs;  see  app.  II  for  a  list  of  airlines  worldwide  that  have 
implemented  foqa  programs.) 


^The  National  Transportation  Safety  Board,  the  official  source  of  information  on  airline  accidents, 
defines  accidents  as  events  in  whidi  individuals  are  killed  or  suffer  serious  tryuiy,  or  the  aircraft  is 
substantially  damaged.  Incidents  are  defined  as  occurrences  other  than  accidents  associated  with  the 
operation  of  an  aircraft  that  affect  or  could  affect  the  safety  of  operations.  49  C.F.R.  830.2. 

^Flight  Safety  Foundation,  Air  Carrier  Voluntary  Fli^t  Operational  Quality  Assurance  Program  (1992). 

^The  flight  data  recorder  is  commonly  referred  to  as  the  “blackbox.” 

^The  six  required  parameters  were  time,  airspeed,  heading,  altitude,  vertical  acceleration,  and  time  of 
radio  transmission. 
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As  part  of  faa’s  strategy  to  achieve  significant  reductions  in  aviation 
accident  rates  despite  the  rapid  increase  in  air  travel  anticipated  over  the 
next  decade,  in  1996  the  agency  initiated  a  foqa  demonstration  project  to 
promote  the  volimtaiy  implementation  of  foqa  programs  by  U.S.  airlines. 
The  objective  of  such  a  program  is  to  use  flight  data  to  detect  technical 
flaws,  imsafe  practices,  or  conditions  outside  of  desired  operating 
procedures  early  enough  to  allow  timely  intervention  to  avert  accidents  or 
incidents.  For  example,  identifying  repeated  instances  of  unstabilized 
approaches  to  a  particular  airport  could  help  to  define  a  new  approach 
pattern  less  likely  to  lead  to  an  accident  under  adverse  conditions,  or  to 
improved  pilot  training.  Such  a  system  has  potentially  broad  application  to 
flight  crews’  performance  and  training,  aircraft  operating  procedures,  air 
traffic  control  procedures,  aircraft  maintenance,  and  airport  design  and 
maintenance.  Major  airlines  in  Europe  and  Asia,  as  well  as  the  U.S.  airlines 
that  have  foqa  programs,  are  uniform  in  their  support  of  the  program. 

How  FOQA  Works,  foqa  involves  (1)  capturing  and  analyzing  flight  data  to 
determine  if  the  pilot,  the  aircraft’s  systems,  or  the  aircraft  itself  deviated 
from  typical  operating  norms;  (2)  identifying  trends;  and  (3)  taking  action 
to  correct  potential  problems.  Airlines  with  foqa  programs  typically  use  a 
device  called  a  quick  access  recorder  to  capture  flight  data  onto  a 
removable  optical  disk  that  facilitates  the  data’s  frequent  removal  firom  the 
aircraft.®  Periodically,  the  optical  disks  are  removed  firom  the  aircraft,  and 
the  flight  data  are  analyzed  by  the  ground  analysis  system  at  a  centralized 
location.  The  data  are  analyzed  by  a  computer  system  that  evaluates  about 
40  to  80  predefined  events  for  deviations  from  the  airbne’s  specified 
tolerance  thresholds.  For  example,  an  event  might  be  the  descent  rate 
during  approach.  Deviations  of  more  than  certain  predetermined  values, 
called  “exceedances,”  are  flagged  and  evaluated  by  a  monitoring  team. 
After  investigating  these  exceedances  to  determine  their  validity  and 
analyzing  them  to  understand  possible  causes,  the  monitoring  team  will 
propose  and  evaluate  corrective  actions.  Periodically,  airlines  aggregate 
exceedances  over  time  to  determine  and  monitor  trends.  (For  a  more 
complete  discussion  of  foqa  operations,  see  app.  I.) 

The  FOQA  Demonstration  Project.  In  July  1996,  faa  initiated  a  3-year, 

$6.6  million  demonstration  project  to  facilitate  the  start-up  of  voluntary 
airline  foqa  programs  and  to  assess  the  costs,  benefits,  and  safety 
enhancement  associated  with  such  programs,  faa  provided  hardware  and 
software  to  each  of  the  three  airlines — ^United,  US  Airways,  and 


^These  data  typically  include  the  parameters  required  to  be  collected  on  the  aircraft’s  flight  data 
recorder  plus  many  more  parameters.  See  app.  I  for  more  information  on  quick  access  recorders  and 
fli^t  data  recorders. 
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Continental— that  have  implemented  poqa  programs  according  to  the 
demonstration  project’s  requirements,  faa purchased  quick  access 
recorders  to  equip  15  Boeing  737  aircraft  at  each  of  the  three  airlines,  faa 
also  purchased  a  ground  analysis  system — the  computer  hardware  and 
software  for  analyzing  and  visualizing  foqa  data — ^for  US  Airways  and 
Continental.  Because  United  already  had  purchased  a  ground  analysis 
system  that  analyzes  these  data  for  other  types  of  aircraft,  faa  purchased 
for  the  airline  the  additional  software  needed  to  analyze  foqa  data  from 
737s.  For  their  part,  these  airlines  funded  the  cost  of  obtaining 
supplemental  type  certification®  of  the  airborne  equipment,  the  costs  of 
installation  and  maintenance,  and  the  cost  of  personnel  to  run  and  monitor 
the  program.  Alaska  Airlines  is  the  fourth  U.S.  airline  to  have  begun  a  foqa 
program,  but  it  has  only  recently  met  the  demonstration  project’s 
requirement  for  an  agreement  on  foqa  by  the  airline’s  pilot  union. 
Consequently,  the  project  has  not  yet  provided  any  equipment  to  the 
airline.  Alaska  Airlines,  however,  received  quick  access  recorders  and  a 
ground  analysis  system  from  the  faa  Structural  Loads  Program  and  uses 
this  equipment  to  operate  its  foqa  program.  (See  app.  Ill  for  more 
information  on  the  Structural  Loads  Program.)  Other  airlines  that  are 
participating  in  the  demonstration  project  and  are  considering  the 
implementation  of  a  foqa  program  are  America  West,  Delta,  Northwest, 
Trans  World,  Southwest,  Continental  Express,  and  United  Parcel  Service.^ 
(See  app.  I  for  a  detailed  description  of  the  faa  demonstration  project.)  As 
a  research  and  development  effort  of  the  foqa  initiative,  faa  is  developing 
the  Aviation  Performance  Measuring  System,  an  advanced  system  for 
conducting  automated  analysis  and  research  on  foqa  data.  (See  app.  Ill  for 
a  description  of  this  system  and  faa’s  other  related  technical  programs.) 

Rather  than  requiring  airlines  to  implement  foqa,  faa  has  chosen  to 
promote  the  initiative  through  a  cooperative  demonstration  project  in 
partnership  with  the  industry.  According  to  the  demonstration  project’s 
program  manager,  it  would  be  premature  for  faa  to  mandate  foqa  at  this 
time  because  U.S.  aviation  is  still  in  the  early  stages  of  developing  foqa 
and  is  primarily  in  a  learning  mode.  The  program  manager  contends  that  a 
mandated  program  would  stifle  innovation,  encounter  substantial 
resistance  from  airlines  and  pilots,  and  most  likely  result  in  minimal 
compliance.  Thus,  at  present,  faa  is  working  with  the  industry  to  raise 
interest  in  the  concept,  facilitate  the  design  and  implementation  of 


®An  FAA  type  certificate  is  issued  when  an  aircraft,  aircraft  engine,  propeller,  or  appliance  is  properly 
designed  and  manufactured,  performs  properly,  and  meets  the  regulations  and  minimum  standards 
prescribed  by  the  Administrator.  An  FAA  supplemental  type  certificate  is  required  when  there  is  a 
change  to  an  aircraft,  aircraft  engine,  propeller,  or  appliance.  49  TJ.S.C.  44704. 

■^Although  not  a  participant  in  the  demonstration  project,  American  Airlines  is  considering  the 
implementation  of  an  internal  FOQA-type  program. 
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voluntary  poqa  programs,  provide  financial  and  technical  assistance,  and 
foster  innovation. 


FOQA  Identifies 
Potential  Safety 
Problems 


The  primary  characteristic  that  distinguishes  foqa  from  other  safety 
reporting  programs,  such  as  the  Aviation  Safety  Reporting  Program  or 
Aviation  Safety  Action  Programs,®  is  that  foqa  provides  objective, 
quantitative  data  on  what  occurs  during  flights  rather  than  what  is 
subjectively  reported  by  individuals.  Instead  of  needing  to  rely  on 
perceived  problems  or  risks,  foqa  yields  precise  information  on  many 
aspects  of  flight  operations,  and  this  information  can  be  used  to  help 
objectively  evaluate  a  wide  range  of  safety-related  issues. 

U.S.  and  foreign  airlines  have  reported  on  previously  unknown  or 
suspected  problems  for  which  foqa  has  provided  objective  information 
that  resulted  in  corrective  actions.  One  airline  found  through  its  foqa 
program  that  more  exceedances  occurred  during  visual  flying  than  during 
instrument  flying.  This  finding  prompted  the  airline’s  flight-training 
managers  to  rethink  the  relative  emphasis  given  visual  and  instrument 
flying  in  the  airline’s  training  programs.  Another  airline’s  foqa  analysis 
confirmed  that  the  incidence  of  descent  rate  exceedances  during 
approaches  was  significantly  higher  at  a  particular  runway  at  a  U.S.  airport 
than  at  other  runways.  After  investigating  the  problem,  the  airline 
concluded  that  the  air  traffic  control  approach  had  been  set  too  high, 
requiring  pilots  to  descend  more  steeply  than  usual  during  their  final 
approach.  When  the  airline  shared  its  findings  with  faa  management,  the 
approach  was  modified  to  correct  this  potential  problem. 

For  landings,  some  airports’  air  traffic  control  procedures  require  pilots  to 
approach  high  and  fast  and  then  descend  steeply.  These  approaches  can 
result  from  a  number  of  factors,  including  noise  abatement  rules,  traffic 
volume,  terrain,  or  weather  conditions.  Although  airline  managers  know 
about  the  situations  from  pilots’  reports,  foqa  gives  them  the  quantitative 
information  to  demonstrate  the  extent  of  this  problem  at  the  various 
airports.  With  these  data  in  hand,  managers  can  be  more  effective  in 
addressing  the  problem  and  taking  action  to  mitigate  or  eliminate  risks. 

FOQA  can  also  help  airlines  determine  the  frequency  of  certain  occurrences 
rather  than  having  to  rely  on  human  judgment,  particularly  for  the  level  of 
maintenance  required.  Two  examples  of  these  types  of  occurrences  are 
hard  landings  and  exceedances  in  engine  temperatures.  Prior  to  foqa. 


®See  app.  IV  for  a  description  of  these  programs. 
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airlines  generally  relied  on  pilots’ judgment  of  the  necessity  for  corrective 
action  if  a  hard  landing  occurred  or  an  engine  overheated,  foqa,  however, 
can  provide  better  information  on  the  amoimt  of  force  the  aircraft 
e35)erienced  during  a  hard  landing.  Similarly,  foqa  gives  more  data  on  the 
engines’  temperatures  and  the  duration  of  overheating  in  some  aircraft 
than  were  previously  available  without  foqa.  With  these  data,  managers 
can  make  more  informed  decisions  about  whether  the  aircraft  needs  to  be 
inspected  to  check  for  structural  damage  or  whether  an  engine  needs  to  be 
overhauled. 

U.S.  and  foreign  airlines  have  reported  that  they  have  used  foqa  analysis  to 
identify  a  variety  of  potential  safety  problems  and  take  corrective  action  to 
resolve  or  mitigate  them.  These  have  included  steep  takeoffs,  which  can 
damage  the  aircraft’s  tail;  approaches  that  are  outside  the  prescribed 
procedures  for  a  “stabilized”  approach;  descent  rates  or  bank  angles  that 
are  considered  excessive;  high  taxi  speeds;  hard  landings;  wind  shear 
occurrences;  ground  proximity  warnings;  and  engine  malfunctions. 
Corrective  action  can  include  notifying  pilots  of  a  change  in  standard 
operating  procedures  or  restating  and  emphasizing  them,  correcting  an 
equipment  problem,  or  providing  additional  training.  The  continued 
monitoring  of  trends  will  tell  the  airline  if  the  corrective  action  has  been 
effective  or  whether  additional  measures  are  needed. 

A  number  of  airlines  plan  to  complement  the  use  of  foqa  data  with 
information  from  safety  reporting  systems,  such  as  Aviation  Safety  Action 
Programs  or  internal  pilot  reporting  systems,  foqa  data,  originating  from 
aircraft  sensors  and  systems,  tell  “what”  happened  to  the  aircraft.  Internal 
safety  reporting  systems,  based  on  reports  of  pilots,  flight  crews,  and  other 
persons,  are  more  likely  to  tell  “why”  something  happened.  Together, 
information  from  foqa  and  internal  reporting  systems  can  provide  valuable 
insight  into  current  and  emerging  problems. 


FOQA’s  Potential 
Costs  and  Benefits 


Based  on  preliminary  estimates  from  an  ongoing  cost-benefit  study  by 
Universal  Technical  Resource  Services,  Inc.  (utrs),  an  faa  contractor, 
table  1  summarizes  the  estimated  annual  costs  for  airlines  to  equip  15,  50, 
and  100  aircraft  with  quick  access  recorders,  purchase  a  ground  analysis 
system,  and  pay  FOQA-related  salaries.® 


^Because  FAA’s  cost-benefit  study  is  in  progress,  we  were  not  able  to  verify  FAA’s  estimates  of  FOQA 
costs  and  savings.  The  cost  and  savings  figures  are  preliminary  and  may  change  as  more  data  are 
gathered. 
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Table  1:  Estimated  Total  Annual  Costs, 

by  Fleet  Size 

15  aircraft 

50  aircraft 

100  aircraft 

Equipment  costs 

$98,500 

$259,000 

$492,000 

Personnel  costs 

385.000 

500.000 

775,000 

Total  annual  costs 

$483,500 

$759,000 

$1,267,000 

Note:  Equipment  costs  are  based  on  the  invoice  price  paid  to  vendors  in  the  FOQA 
demonstration  project  To  annualize  the  figures,  the  equipment  purchase  costs  have  been  spread 
over  a  5-year  period.  Personnel  costs  are  based  primarily  on  estimates  of  FOQA  management, 
analysis,  monitoring,  and  engineering  costs  from  an  airline  participating  in  the  demonstration 
project. 

Source:  UTRS. 


The  cost-benefit  study  estimates  that  airlines  will  reduce  their 
expenditures  for  fuel  and  maintenance  as  well  as  reduce  the  number  of 
accidents  and  incidents  over  time,  avoiding  their  associated  costs. 
Because  foqa  programs  analyze  additional  data  on  aircraft  systems  and 
engine  conditions,  airlines  are  better  able  to  achieve  optimum  fuel 
consumption  and  avoid  unneeded  engine  maintenance.  Although  more 
difficult  to  quantify  and  directly  relate  to  a  foqa  program,  enhanced  safety 
should  result  in  lower  costs  over  time  as  a  result  of  accidents  avoided  and 
lower  insurance  premiums.  Table  2  summarizes  the  estimated  annual 
savings  for  fleet  sizes  of  15,  50,  and  100  aircraft.  Fuel  savings  and  engine 
savings  figures  are  based  on  estimates  of  a  0.5-percent  reduction  in  fuel 
consumption  and  a  1-percent  reduction  in  engine  maintenance  costs.  The 
safety  savings  figure  is  based  on  a  hypothetical  1-percent  reduction  in  the 
annual  costs  incurred  from  accidents,  faa’s  contractor  based  its  safety 
savings  calculation  on  a  current  loss  rate  of  2  aircraft  per  million 
departures  at  a  cost  of  $160  million  for  each  loss. 
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Savings,  by  Fleet  Size 

15  aircraft 

50  aircraft 

100  aircraft 

Fuel  savings 

$145,800 

$486,000 

$972,000 

Engine  savings 

300,000 

1,000,000 

2,000,000 

Safety  Savings 

49,500 

165,000 

330,000 

Total  annual  savings 

$495,300 

$1,651,000 

$3,302,000 

Note:  Fuel  and  engine  savings  were  estimated  on  the  basis  of  discussions  with  an  airline 
participating  in  the  FOQA  demonstration  project.  Safety  savings  were  estimated  on  the  basis  of 
information  from  a  European  airline  with  a  long-term  FOQA  program.  Savings  estimates  were  also 
based  on  an  assumption  of  3,000  flight  hours  per  aircraft  per  year. 

Source:  UTRS. 


According  to  these  annual  cost  and  savings  estimates,  foqa  would  result  in 
net  annual  savings  of  $11,800  for  15  aircraft,  $892,000  for  50  aircraft,  and 
$2,035,000  for  100  aircraft.  See  table  3. 


Savings,  by  Fleet  Size 

15  aircraft 

50  aircraft 

100  aircraft 

Total  annua!  costs 

$483,500 

$759,000 

$1,267,000 

Total  annual  savings 

495,300 

1,651,000 

3,302,000 

Net  annual  savings 

$11,800 

$892,000 

$2,035,000 

Source:  UTRS. 


Factors  Impeding 
Implementation  and 
Actions  to  Overcome 
Impediments 


Although  airline  officials,  pilot  organizations,  and  faa  officials  recognize 
the  potential  for  improving  safety  and  operations  through  foqa  programs, 
airline  officials  and  representatives  of  the  pilot  organizations  were 
unanimous  in  their  view  that  data  protection  issues  need  to  be  resolved. 
Both  airline  officials  and  pilots’  representatives  stated  that  the  lack  of 
protections  for  foqa  data  has  been  a  mjgor  contributor  to  pilot  unions’ 
reluctance  to  sign  foqa  agreements  with  airlines  and  airlines’  reluctance  to 
implement  foqa  programs. 


According  to  the  Flight  Safety  Foundation’s  1992  report,  the  greatest 
impediment  to  the  implementation  of  foqa  in  the  United  States  is 
associated  with  the  “protection  of  data  firom  use  for  other  than  safety  and 
operational  improvement  purposes.”  Basically,  airline  managers  and  pilots 
have  three  concerns:  (1)  that  the  information  may  be  used  in 
enforcement/discipline  actions,  (2)  that  such  data  in  the  possession  of  the 
federal  government  may  be  obtained  by  the  public  and  the  media  through 
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Enforcement 


the  provisions  of  the  Freedom  of  Information  Act  (foia),  and  (3)  that  the 
information  may  be  obtained  in  civil  litigation  through  the  discovery 
process.  Similar  concerns  have  been  expressed  in  connection  with  other 
programs  under  which  information  is  submitted  voluntarily  to  faa. 


Representatives  from  each  of  the  major  airlines  as  well  as  the  imions  that 
represent  pilots  from  the  major  airlines — the  Air  Line  Pilots  Association, 
the  Allied  Pilots  Association,  the  Independent  Association  of  Continental 
Pilots,  and  the  Southwest  Airlines  Pilot  Association— told  us  that  the 
airlines  and  pilots  fear  the  possibility  that  foqa  data  might  be  used  against 
them  in  faa  enforcement  proceedings.  In  addition  to  these  concerns, 
pilots’  representatives  were  concerned  that  airline  managers  could  use 
FOQA  data  to  punish  or  discipline  pilots. 

FAA  Enforcement.  Many  U.S.  airlines  and  their  pilots  appear  frustrated  with 
faa’s  delay  in  issuing  a  regulation  implementing  the  nonenforcement 
policy  articulated  in  a  February  1996  policy  letter  from  the  Administrator 
to  the  Air  line  Pilots  Association  and  the  Air  Transport  Association,  faa’s 
letter  said  that  no  enforcement  action  will  be  taken  on  the  basis  of  the 
information  gained  through  foqa.  Specifically  the  letter  stated: 


“The  FAA  commits  that  it  will  not  use  information  collected  by  a  carrier  in  an  foqa  program 
to  imdertake  any  certificate  or  other  enforcement  action  against  an  air  carrier  participating 
in  such  a  program  or  one  of  its  individual  employees.  Notwithstanding^  the  faa  reserves  its 
rights  to  use,  for  any  other  purpose,  information  obtained  from  sources  other  than  foqa, 
including  flight-recorder  parameters  speciftcally  required  by  the  Federal  Aviation 
Regulations.  The  limitation  on  the  use  of  information  applies  only  to  information  collected 
specifically  in  an  foqa  program.” 

In  an  April  1997  letter  to  the  Air  Transport  Association’s  foqa  Steering 
Committee,  the  Director  of  faa’s  Flight  Standards  Service  said  that  the 
1995  policy  letter  wiU  remain  in  effect  until  the  regulation  on  enforcement 
is  issued.  The  letter  stated  that  a  proposed  rulemaking  setting  forth  faa’s 
enforcement  protection  policy  should  be  ready  by  the  end  of  1997. 

According  to  airline  officials  and  a  pilot  union’s  representative,  faa’s  delay 
in  promulgating  an  enforcement  regulation  has  hampered  efforts  to  reach 
agreement  with  some  pilot  imions  and  threatens  the  continuance  of 
agreements  already  reached.  One  of  the  issues  facing  faa  is  how  broad  the 
enforcement  protection  should  be.  faa  attorneys  have  concluded  that  it  is 
beyond  the  scope  of  faa’s  authority  and  in  violation  of  its  statutory  duties 
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to  issue  a  regulation  that  precludes  the  agency  from  taking  action  if  foqa 
data  reveal  that  an  airplane  was  not  in  a  condition  for  safe  flight  or  that  a 
pilot  lacked  qualifications.  Pilots’  representatives,  however,  have  cited  the 
precedent  of  faa’s  cockpit  voice  recorder  regulation  that  prohibits  the 
agency  from  using  the  record  in  enforcement  actions  without  exceptions. 


FAA  officials  told  us  that  the  agency  is  trying  to  find  the  proper  balance 
between  carrying  out  its  enforcement  responsibilities  and  providing 
incentives  for  implementing  safety  programs  and  sharing  information  with 
FAA.  In  similar  programs,  such  as  the  Aviation  Safety  Reporting  Program, 
Air  Carrier  Volimtary  Disclosure  Reporting  Procedures,  and  Aviation 
Safety  Action  Programs,”  imder  which  safety  information  is  volimtarily 
submitted,  the  agency  has  a  policy  of  addressing  alleged  violations  through 
administrative  actions  or  forgoing  and/or  waiving  the  imposition  of  any 
legal  enforcement  if  certain  qualifying  criteria  are  met.  These  programs  are 
intended  to  encourage  prompt  reporting  of  violations,  sharing  of  important 
safety  information,  and  pilot  training  to  enhance  future  compliance.  While 
the  qualifying  criteria  differ  for  each  program,  these  programs  exclude 
actions  that  are  deliberate  or  demonstrate  or  raise  questions  of 
qualifications.  Generally,  the  parameters  of  the  programs,  including  the 
qualifying  criteria,  are  spelled  out  in  the  governing  advisory  circular.  It  is 
faa’s  belief  that  by  offering  incentives,  such  as  forgoing  legal  enforcement 
actions  under  certain  conditions,  more  problems  may  be  reported  and 
ultimately  corrected  than  could  be  discovered  through  other  means,  such 
as  inspections. 

Airline  Enforcement.  Airline  managers  are  working  with  their  respective 
pilot  unions  to  enter  into  data-use  agreements  that  include  individual 
protection  provisions.  According  to  the  Flight  Safety  Foundation  study, 
data-use  agreements  with  pilot  associations  have  existed  since  flight  data 
recorders  were  first  required  in  the  late  1950s.  Having  such  an  agreement 
is  a  precursor  to  becoming  a  full  partner  in  the  foqa  demonstration 
project.  Generally,  these  agreements  provide,  among  other  things,  the 
company’s  assurance  not  to  use  the  recorded  flight  data  for  punitive  or 


cockpit  voice  recorder  regulation  provides  that:  “The  Administrator  does  not  use  the  cockpit 
voice  recorder  record  in  any  ci^  penalty  or  certificate  action.”  14  C.F.R.  91.609(g).  FAA*s  regulations 
also  provide  enforcement  protection  wi^  some  qualifications  to  information  collected  under  the 
Aviation  Safety  Reporting  Program.  Specifically,  the  regulation  provides  that  “The  Administrator  of  the 
FAA  will  not  use  reports  submitted  to  the  National  Aeronautics  and  Space  Administration  under  the 
Aviation  Safety  Reporting  Program  (or  information  derived  therefrom)  in  any  enforcement  action 
except  information  concerning  accidents  or  criminal  offenses  which  are  wholly  excluded  from  the 
Program.”  14  C.F.R.  91.26. 

^^See  app.  IV  for  a  description  of  these  programs. 
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disciplinary  action  against  a  crew  member,  or  as  evidence  in  any 
proceeding.  Also,  to  ensure  the  protection  of  the  company’s  employees, 
the  data-use  agreements  generally  provide  for  the  de-identification  of  the 
information  as  soon  as  possible,  usually  within  7  days.  This  practice 
ensures  the  confidentiality  and  anonymity  of  the  flight  crew  members 
participating  in  the  program. 


Freedom  of  Information 
Act  Requests 


Both  airlines  and  pilots  are  concerned  that  foqa  data  could  become  public 
and  available  to  the  media  through  the  federal  foia,  if  such  data  are 
provided  directly  to  faa.^^  The  federal  foia  sets  forth  a  policy  of  broad 
disclosure  of  government  documents  to  ensure  “an  informed  citizenry, 
vital  to  the  functioning  of  a  democratic  society.”  NLRB  v.  Robbins  Tire  & 
Rubber  Co.,  437  U.S.  214, 242  (1978).  The  Congress  understood,  however, 
that  “legitimate  governmental  and  private  interest  could  be  harmed  by 
release  of  certain  types  of  information.”  FBI  v.  Abramson,  456  U.S.  616,  621 
(1982).  Accordingly,  the  act  provides  for  nine  categorical  exemptions. 

In  the  past,  safety  information  voluntarily  submitted  to  faa,  for  example 
under  Air  Carrier  Voluntary  Disclosure  Reporting  Procedures,  has  been 
protected  under  exemption  4  of  foia.  Exemption  4  protects  trade  secrets 
and  commercial  or  financial  information  obtained  from  a  person  that  is 
privileged  or  confidential  Airline  officials  and  pilots’  representatives 
expressed  concern  that  foqa  data  may  not  be  protectable  under  this 
exemption. 

Recently,  the  Congress  enacted  the  Federal  Aviation  Reauthorization  Act 
of  1996,  which  contains  a  provision  that  protects  voluntarily  submitted 
information  under  certain  circumstances.  Specifically,  under  the 
provision,  notwithstanding  any  other  provision  of  law,  the  Administrator  is 
barred  from  disclosing  voluntarily  provided  safety-  or  security-related 
information  if  the  Administrator  finds  that 

“(1)  the  disclosure  of  the  information  would  inhibit  the  voluntary  provision  of  that  type  of 
information  and  that  the  receipt  of  that  type  of  information  aids  in  fulfilling  the 
Administrator’s  safety  and  security  responsibilities;  and  (2)  withholding  such  information 
from  disclosure  would  be  consistent  with  the  Administrator’s  safety  and  security 
responsibiUties.”  49  U.S.C.  40123.1^ 


^^Currently,  airlines  provide  no  FOQA  data  to  FAA.  Rather,  FAA  reviews  aggregated  trend  information 
on  the  airlines’  premises. 

^®A  similar  provision  was  included  in  the  National  Transportation  Safety  Board  Amendments  of  1996  to 
protect  information  that  is  voluntarily  submitted  to  the  Board. 
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The  provision  also  requires  the  Administrator  to  issue  regulations  to 
implement  the  section. 

The  House  report  accompanying  this  legislation  noted  with  approval  the 
data-sharing  programs  such  as  foqa  and  the  Committee’s  intent  to 
encourage  and  promote  these  sorts  of  innovative  safety  programs.  The 
report  provides  that  information  submitted  under  these  programs  would 
arguably  be  protected  from  release  under  exemption  4  of  foia;  however, 
the  report  notes  that  such  a  decision  to  withhold  the  information  would  be 
discretionary  with  the  agency.  The  report  states  that  to  provide  assurance 
that  such  information  is  not  publicly  released,  the  legislation  woidd 
prohibit  faa  from  disclosing  voluntarily  submitted  safety  information. 
According  to  the  report,  this  protection  should  “alleviate  the  aviation 
community’s  concerns  and  allow  the  data-sharing  safety  programs  to  move 
forward.”  Moreover,  the  report  noted  that  the  provision  would  not  reduce 
the  information  available  to  the  public,  since  the  public  does  not  receive 
the  data.  Rather,  the  report  states  that  public  safety  will  be  enhanced  by 
the  increase  in  faa’s  understanding  of  ongoing  trends  in  operations  and 
technologies.^^ 

FAA  is  currently  working  on  a  rulemaking  procedure  that  will  prohibit  the 
release  of  voluntarily  submitted  safety  data  through  foia.^®  It  is  expected 
that  the  rulemaking  will  provide  the  procediures  that  the  agency  will  use  in 
making  the  required  determinations.  It  is  also  expected  that  foqa  data  will 
be  proposed  as  qualifying  for  the  protection.  According  to  an  faa  attorney, 
the  determinations  for  the  foqa  program  may  be  included  in  the  notice  of 
proposed  rulemaking  on  the  foqa  nonenforcement  policy.  The  anticipated 
FOIA  rulemaking  and  the  subsequent  findings  to  include  the  foqa  program 
within  the  protection  should  help  mitigate  or  resolve  the  industry’s  fears 
about  the  possible  disclosure  of  foqa  data  through  foia  requests  if  foqa 
data  are  provided  directly  to  faa. 


Discovery  Process  in  Civil 
Litigation 


Some  airline  officials  have  told  us  that  although  they  want  to  improve 
aviation  safety  by  implementing  a  foqa  program,  the  voluntary  collection 


“H.R.  Rep,  No.  104-714  pt  1  at  pp.  4041  (1996). 

the  Final  Report  of  the  White  House  Commission  on  Aviation  Safety  and  Security,  dated  Feb.  12, 
1997,  a  recommendation  was  made  that  FAA  should  work  with  the  aviation  commimity  to  develop  and 
protect  the  integrity  of  standard  safety  databases  that  can  be  shared  in  accident  prevention  programs. 
The  report  stated  that  FAA  needed  to  expeditiously  complete  rulemaking  to  implement  the  voluntary 
disclosure  protection  provision  and  that  the  agency  should  assess  the  adequacy  of  the  new  legislative 
authority  and  implementing  regulation  1  year  after  the  regulations  take  effect.  The  report  further 
stated  that  any  necessary  regulatory  or  legislative  modifications  identified  at  that  time  should  be 
promptly  addressed. 
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of  data  may  potentially  expose  airlines  to  greater  liability  in  civil  litigation. 
POQA  data  may  indicate  conditions  outside  of  desired  operating 
procedures.  Airline  officials  and  pilot  representatives  told  us  that  they  are 
concerned  that  through  broad  discovery  rules,  foqa  data  could  be 
inappropriately  used  or  disclosed  to  the  public.  The  general  purpose  of 
discovery  is  to  remove  surprise  from  trial  preparation  so  that  parties  may 
obtain  the  evidence  necessary  to  evaluate  and  resolve  their  dispute.^® 

Since  foqa  data  are  retained  at  the  airlines  and  are  not  currently  provided 
directly  to  faa,  the  focus  has  been  on  the  airlines’  ability  to  protect  the 
information. 

Under  federal  rules,  parties  in  litigation  in  federal  court  are  authorized  to 
obtain  discovery  of  any  matter,  not  privileged,  which  is  relevant  to  the 
subject  matter  involved  in  the  pending  action,  whether  it  relates  to  the 
claim  or  defense  of  the  party  seeking  discovery  or  to  the  claim  or  defense 
of  any  other  party.  Generally,  privileges  are  narrowly  construed  and  in 
some  cases  are  qualified.  However,  even  in  the  absence  of  a  privilege,  a 
district  court  has  broad  discretion  imder  the  federal  rules  to  issue  an  order 
to  protect  a  person  from  annoyance,  embarrassment,  oppression,  or  undue 
burden  or  expense  if  there  is  a  good  cause  for  issuance  of  the  order. 

Courts  generally  invoke  a  balancing  test  to  decide  when  a  protective  order 
is  appropriate  and  how  it  is  to  be  applied. 

In  two  recent  cases,  the  airlines  have  tried  to  convince  federal  courts  that 
voluntarily  collected  safety  data  similar  to  foqa  data  should  be  protected 
from  discovery  or,  at  the  very  least,  covered  under  a  protective  order.  In 
both  cases,  the  courts  sought  to  achieve  a  balance  between  the  airlines’ 
desire  to  protect  the  information  and  the  plaintiffs’  right  to  a  fair  trial.  In 
the  first  case,  the  court  rejected  a  claim  that  the  information  should  be 
protected  imder  the  self-critical  evaluation  privilege  but  limited  the 
possible  uses  of  the  documents  it  ordered  to  be  produced.^®  This 
determination  was  effected  through  a  protective  order.  In  the  other  case. 


Moore*s  Federal  Practice,  section  26.02  (Matthew-Bender  3d  ed.). 

^"^6  Moore's  Federal  Practice,  ch.  26  (Matthew-Bender  3d  ed.). 

^®Court  Order  of  Oct.  26, 1995,  In  re:  Air  Crash  at  Charlotte,  North  Carolina,  on  July  2, 1994,  MDL 
Docket  No.  1041  (D.S.C.  1996)  (the  court  rejected  the  claim  of  self-critical  evaluation  privilege);  but 
see,  Court  Order  of  Nov.  14, 1995,  In  re:  Air  Crash  at  Charlotte,  North  Carolina,  on  July  2  1994,  MDL 
Docket  No.  1041  (D.S.C.  1996)  (the  court  issued  a  protective  order);  and  In  re:  Air  Crash  Near  Cali, 
Columbia,  on  Dec.  20, 1996,  969  F.  Supp.  1529  (S.D.  Fla.  1997)  (the  court  rejected  the  claim  of 
self-critical  evaluation  privilege  but  recognized  a  new  qualified  privilege  for  the  American  Airlines 
Safety  Action  Program).  For  a  more  detailed  discussion  of  these  court  cases,  see  app.  V. 

^^e  self-critical  evaluation  privilege,  when  recognized,  protects  documents  that  reflect  an  internal 
self-analysis. 
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the  court  also  rejected  the  claim  of  self-critical  evaluation  privilege  but  at 
the  same  time  recognized  a  new  qualified  privilege  for  information 
collected  under  a  partnership  program  with  faa,  the  American  Airlines 
Safety  Action  Program.^® 

Although  airlines  are  generally  pleased  with  the  coiut’s  decision  to  grant  a 
qualified  privilege  to  Aviation  Safety  Action  Program  materials,  it  is  not 
clear  whether  other  courts  will  recognize  this  new  privilege  or  extend  it  to 
other  safety  and  security  information  that  has  been  voluntarily  collected. 
Nor  is  there  a  guarantee  that  foqa  data  or  other  similar  information,  if 
foimd  not  to  be  privileged,  would  be  covered  under  a  protective  order. 
However,  we  found  no  instances  to  date  in  which  foqa  data  have  been 
subject  to  a  discovery  request.  This  situation  may  result  firom  the  fact  that 
airlines  are  just  beginning  to  institute  foqa  programs.  However,  some  of 
the  pilot  union  officials  we  spoke  with  noted  that  discovery  is  a  concern 
because  of  the  potentially  large  amounts  of  data  that  wiU  be  collected. 
While  some  in  the  aviation  community  beheve  that  one  way  to  ensure 
protection  would  be  through  legislation,  there  does  not  appear  to  be  a 
consensus  to  seek  legislation  at  this  time.^‘  Concern  has  been  expressed 
that  the  failure  of  a  legislative  effort  may  adversely  affect  how  courts  treat 
voluntarily  collected  safety  information. 

In  the  event  that  faa  does  receive  foqa  data  directly,  according  to  faa 
attorneys,  it  has  provisions  in  place  for  dealing  with  requests  firom  private 
litigants  for  documents  in  the  agency’s  possession,  faa  attorneys  noted 
that  a  request  for  records  from  a  private  litigant,  when  the  agency  is  not  a 
party  to  the  action,  wifi  generally  be  treated  as  a  foia  request  (see  49 
C.F.R.  9.13).  If  the  agency  is  a  party  to  the  litigation,  faa  will  seek  to 
protect  the  information,  if  appropriate,  under  a  claim  of  government 
privilege  and,  if  that  fails,  to  release  the  information  xmder  a  protective 
order. 


Agency  Comments 


We  provided  copies  of  a  draft  of  this  report  to  the  Department  of 
Transportation  and  faa  for  their  review  and  comment.  We  met  with 
officials,  including  faa’s  Deputy  Associate  Administrator  for  Regulation 
and  Certification  and  the  demonstration  project’s  program  manager.  They 


the  court  provided  that  the  plaintiff  could  come  forward  with  a  persuasive  showing  of  need  and 
hardship.  In  such  case,  the  court  would  review  the  voluntarily  coEected  information  in  camera  and 
evaluate  whether  the  plaintiff’s  interests  overcome  the  powerful  interest  that  weighs  in  favor  of 
preserving  the  confidentiality  of  the  information.  No  such  showing  was  made  in  this  case. 

^^Limited  legislative  protection  has  been  provided  for  cockpit  voice  recorders,  49  U.S.C.  1164. 
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agreed  with  the  report  and  provided  several  technical  corrections,  which 
were  incorporated  into  the  report. 


Scope  and 
Methodology 


To  obtain  the  information  in  this  report,  we  reviewed  faa’s  foqa 
demonstration  project’s  requirements,  policies,  and  plans  to  assist  airlines 
in  implementing  foqa  programs.  We  discussed  specific  details  of  the 
project  with  faa’s  Deputy  Associate  Administrator  for  Regulation  and 
Certification  as  well  as  the  demonstration  project’s  program  manager  and 
contractor.  We  conducted  interviews  with  faa  and  National  Aeronautics 
and  Space  Administration  officials  responsible  for  developing  the  Aviation 
Performance  Measuring  System.  We  discussed  foqa  issues  with  the 
National  Transportation  Safety  Board.  We  interviewed  representatives  of 
each  of  the  10  largest  passenger  airlines:  Alaslm,  America  West,  American, 
Continental,  Delta,  Northwest,  Southwest,  Trans  World,  United,  and  US 
Airways;  representatives  of  each  of  the  four  unions — ^the  Air  Line  Pilots 
Association,  the  Allied  Pilots  Association,  the  Independent  Association  of 
Continental  Pilots,  and  Southwest  Airlines  Pilot  Association — ^representing 
the  pilots  of  these  airlines;  and  United  Parcel  Service.  We  also  conducted 
interviews  with  the  Air  Transport  Association,  the  Flight  Safety 
Foundation,  and  the  vendors  providing  hardware  and  software  for  the 
demonstration  project.  Last,  we  interviewed  and  collected  information 
from  foreign  airlines  and  Britain’s  Civil  Aviation  Authority  on  their 
respective  foqa  efforts. 

We  performed  our  work  primarily  at  faa  headquarters  in  Washington,  D.C., 
and  conducted  our  evaluation  from  January  through  October  1997  in 
accordance  with  generally  accepted  government  auditing  standards. 


As  requested,  we  plan  no  further  distribution  of  this  report  until  30  days 
after  the  date  of  this  letter  unless  you  publicly  announce  the  report’s 
contents  earlier.  At  that  time,  we  will  send  copies  to  the  appropriate 
congressional  committees  and  the  Department  of  Transportation  and  faa. 
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We  will  also  make  copies  available  to  others  upon  request.  Please  call  me 
at  (202)  512-2834  if  you  have  any  questions  about  this  report.  Major 
contributors  to  this  report  are  listed  in  appendix  VI. 


John  H.  Anderson,  Jr. 

Director,  Transportation  Issues 
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Abbreviations 

APMS  Aviation  Performance  Measuring  System 

AQP  Advanced  Qualification  Program 

ASAP  Aviation  Safety  Action  Programs 

ASRP  Aviation  Safety  Reporting  Program 

ASES  Aviation  Safety  Reporting  System 

BA  British  Airways 

DEMOPROJ  FOQA  demonstration  project 

DFDAU  digital  flight  data  acquisition  unit 

FAA  Federal  Aviation  Administration 

FAR  Federal  Aviation  Regulation 

FOiA  FVeedom  of  Information  Act 

FOQA  Flight  Operational  Quality  Assurance 

FSF  Flight  Safety  Foundation 

GAIN  Global  Analysis  and  Information  Network 

KLM  Royal  Dutch  Airlines 

NASA  National  Aeronautics  and  Space  Administration 

PCMCIA  Personal  Computer  Memory  Card  International 

Association 

QAE  quick  access  recorder 

UTRS  Universal  Technical  Resource  Services,  Inc. 
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FOQA’s  Background  Right  Operational  Quality  Assurance  (foqa)  had  its  origin  in  the  use  of 

flight  data  recorders  as  mandated  by  the  Civil  Aeronautics  Administration 
in  1958.  Although  the  first  flight  data  recorders  captured  only  six 
parameters — ^time,  airspeed,  heading,  altitude,  vertical  acceleration,  and 
time  of  radio  transmission — ^they  were  a  valuable  tool  for  reconstructing 
what  had  occurred  before  and  during  accidents.  By  the  1960s,  airlines  had 
begun  to  monitor  data  on  routine  flights.  Initially,  the  monitoring  systems 
captured  airworthiness  data,  but  over  time  they  have  expanded  to  include 
operational  data.  In  the  late  1960s,  Trans  World  Airlines  began  a  program 
to  monitor  a  limited  number  of  parameters  related  to  approaches  and 
landings  as  flight  data  recorders  received  periodic  maintenance. 

At  least  eight  foreign  airlines  have  had  FOQA-type  programs  in  operation  for 
over  25  years.  A  program  using  data  from  flight  data  recorders  was  begun 
by  British  Airways  (ba)  in  1962  to  validate  airworthiness  criteria.  Although 
limited  by  today’s  standards,  ba’s  program  contained  the  seeds  of  a 
modem,  safety-oriented  foqa  program.  Ciurrently,  ba  analyzes  the  flight 
data  from  all  of  the  aircraft  in  its  fleet  through  its  Special  Events  Search 
and  Master  Analysis  program.  Over  the  years,  the  number  of  foreign 
airlines  that  have  implemented  a  FOQA-type  program  has  steadily  risen. 
Japan  Airlines’  foqa  program  of  over  15  years  includes  a  printer  in  the 
cockpit  so  that  pilots  can  monitor  their  own  performance.  All  Nippon 
Airways  began  a  program  to  analyze  flight  data  in  1974.  Other  foreign 
airlines  with  established  foqa  programs  include  Scandinavian  Airlines 
System,  Royal  Dutch  Airlines  (klm),  and  Lufthansa.  Many  of  these  airlines 
are  convinced  that  foqa  is  a  critical  component  in  their  safety  efforts  and 
that  the  program  has  paid  valuable  safety  dividends  over  the  years. 
Currently,  about  33  foreign  airlines  and  4  U.S.  airlines — ^United,  US 
Airways,  Continental,  and  Alaska— have  implemented  foqa  programs  (see 
app.  n  for  the  complete  list). 

Recognizing  the  value  of  operational  flight  data  and  the  critical  nature  of 
flight  crews’  performance  in  accidents,  the  Flight  Safety  Foundation  (fsf) 
proposed  and  was  selected  by  the  Federal  Aviation  Administration  (faa)  in 
1991  to  study  foqa.  In  its  1992  report  on  foqa,  fsf  said  that 


“The  proposal  was  based  on  fsf's  conviction,  formed  by  the  positive  experiences  of  its 
international  member  airlines  using  foqa,  that  the  appropriate  use  of  foqa  data  by  airlines, 
pilot  associations  and  aircraft  and  equipment  manufacturers  would  result  in  a  significant 
improvement  of  flight  safety  by  identifying  operational  irregularities  that  can  foreshadow 
accidents  and  incidents.” 
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How  FOQA  Works 


The  FSF  study  concluded  that  foqa  must  proceed  in  the  United  States  and 
that  the  implementation  of  foqa  by  U.S.  airlines  would  have  a  more 
positive  impact  on  Part  121  operational  safety  than  any  other  human 
factors  program  included  in  faa’s  research  and  development  plans,  fsf 
recommended  that  faa  promote  voluntary  foqa  programs  by  instituting  a 
demonstration  program  in  partnership  with  industry.  In  1992,  faa’s  Flight 
Standards  Service  proposed  funding  for  a  demonstration  program.  On 
February  9, 1995,  faa  announced  its  plans  for  an  FAA-industry 
demonstration  project,  and  the  Administrator  sent  a  policy  letter  to  the  Air 
Transport  Association  and  the  Air  line  Pilots  Association  stating  that  faa 
would  not  use  foqa  data  for  enforcement  purposes,  provided  that  the 
airlines  met  certain  requirements. 


At  a  minimum,  foqa  involves  the  analysis  of  flight  data  on  a  routine  basis 
to  reveal  situations  requiring  corrective  actions  before  problems  occur.  To 
institute  such  a  program,  airlines  need  methods  to  capture  flight  data, 
transform  the  data  into  the  appropriate  format  for  analysis,  and  generate 
reports  and  visualizations  to  assist  personnel  in  analyzing  the  data. 
Although  different  methods  are  available,  the  following  describes  how  a 
representative  foqa  program  operates;  the  descriptions  are  based  on  the 
experience  of  the  four  U.S.  airlines  that  have  implemented  foqa. 

Management.  A  typical  program  is  managed  and  operated  by  a  foqa 
manager,  one  or  more  analysts,  and  a  foqa  monitoring  team  (sometimes 
referred  to  as  the  exceedance  guidance  team)  made  up  of  airline  pilots 
who  work  on  foqa  on  a  part-time  basis.  Generally,  the  majority  of  the 
monitoring  team’s  pilots  are  also  representatives  of  the  pilot  union.  These 
individuals  manage  the  foqa  program  in  strict  adherence  to  the 
agreements  made  with  the  pilot  union,  most  notably  on  ensuring  the 
confidentiality  of  pilots’  identities.  This  group  is  responsible  for  defining 
and  refining  exceedances  and  parameters,  reviewing  and  analyzing  data, 
and  determining  and  monitoring  corrective  actions. 

Data  capture.  The  first  step  is  the  capture  of  data  over  the  duration  of  the 
flight.  FlighFdata  comprise  snapshots  of  values  or  measurements  from 
various  aircraft  systems.  Each  data  item  represents  information  from  a 
discrete  source,  such  as  an  instrument  or  sensor.  Generally,  these  data 
items  are  referred  to  as  “parameters.”  Examples  of  parameters  are 
“altitude”  or  “landing  gear  position.”  Recording  rates  vary,  depending  on 
the  parameter,  ranging  from  many  times  per  second  to  about  once  per 
minute. 
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Although  flight  data  recorders  continuously  record,  at  a  minimum, 
FAA-mandated  parameters  during  every  flight,  they  typically  are  not 
designed  to  provide  frequent  access  to  their  data  but  rather  to  survive  the 
extreme  conditions  during  and  after  crashes  to  preserve  flight  data  for 
accident  investigations.  These  devices  are  housed  in  crash-resistant, 
sealed  containers  designed  to  Avithstand  high  “g”  forces,  submersion  in 
water,  and  fire.  Obtaining  frequent  access  to  flight  data  recorders  for  foqa 
purposes,  however,  would  produce  increased  wear  on  internal 
mechanisms  and  result  in  shortened  mechanical  life  and  increased 
expense  for  a  very  specialized  device.^^  Also,  flight  data  recorders  may  not 
capture  a  sufficient  number  of  parameters  to  be  useful  for  foqa  purposes. 
Currently,  faa  requires  from  16  to  29  parameters  to  be  recorded  on  flight 
data  recorders  in  transport  aircraft;^^  a  foqa  program,  however,  would 
likely  capture  many  more  parameters.  Typically,  the  200-600  parameters 
available  on  modem  digital  aircraft  allow  a  more  comprehensive  set  of 
conditions  to  be  monitored.  Finally,  flight  data  recorders  hold  about  25 
hours  of  flight  data,  a  relatively  short  time  period.  Instead,  some  U.S. 
airlines  use  a  device  called  a  quick  access  recorder  (qar)  to  record  foqa 
data  to  a  removable  optical  disk  or  Personal  Computer  Memory  Card 
International  Association  (pcmcia)  card.^  qars  record  flight  data  that  are 
output  from  the  aircraft’s  digital  flight  data  acquisition  unit  (dfdau),  the 
same  device  that  feeds  parameters  to  the  flight  data  recorder.  On  average, 
QAEs  hold  from  100  to  200  hours  of  flight  data. 

Data  transfer.  As  aircraft  receive  periodic  servicing,  the  medium  (optical 
disk  or  pcmcia  card)  containing  flight  data  is  removed  from  the  qar  and 
sent  to  a  central  location  for  analysis.  A  new  disk  or  card  is  inserted  into 
the  QAR  for  the  next  round  of  flights.  Airlines  retrieve  the  data  on 
schedules  ranging  from  3  to  20  days. 

An  alternative  to  physical  recording  media  is  the  use  of  datalink  systems  to 
transmit  information  directly  to  the  ground-based  system,  eliminating  the 
need  to  retrieve  data  from  the  aircraft.  Two  participating  airlines  are 
investigating  the  use  of  automatic  wireless  data  transfer  upon  landing  at 
specially  equipped  airports.  Data  would  be  transmitted  on  a  radio 


^^The  newer  solid-state  flight  data  recorders,  however,  have  no  moving  parts  and  would  not  experience 
wear  problems.  Transferring  data  from  these  devices  takes  several  minutes  to  perform. 

^^nder  a  recently  issued  rule,  FAA  requires  the  recording  of  16  to  29  parameters  by  the  flight  data 
recorder  on  all  existing  transport  aircraft,  depending  on  the  aircraft  model,  its  internal  systems,  and  its 
date  of  manufacture.  Aircraft  manufactured  after  the  rule,  however,  will  be  required  to  record  88 
parameters  within  6  years.  62  Fed.  Reg.  38362  (July  17,  1997). 

^^Other  airborne  data  collection  systems  in  use  aroimd  the  world  include  QARs  using  tape  cartridges 
and  solid-state  devices. 
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frequency  link  from  the  aircraft  to  a  receiving  station  after  the  aircraft 
lands.  In  turn,  a  local  area  network  would  transfer  the  data  to  the  ground 
analysis  station.  Data  encryption  and  other  methods  would  be  used  to 
ensure  the  security  of  the  transmitted  foqa  data. 

Data  processing  and  analysis.  Each  airline  has  a  ground  analysis  system 
where  airborne  collected  data  are  processed  and  analyzed.  The  ground 
analysis  system  transforms  the  raw  digital  flight  records  into  usable  form 
for  processing,  analyzes  the  flight  information,  and  generates  information 
on  any  detected  exceedances  that  represent  deviations  from  normal 
operating  practices  or  exceptional  conditions. 

The  flight  data  analysis  component  of  the  ground  analysis  system 
categorizes  operational  events  to  be  flagged  by  defining  a  set  of 
parameters  that  indicate  normal  operating  envelopes.  The  associated 
thresholds  for  these  parameters  vary  by  the  type  of  aircraft  and  associated 
operating  limits,  accepted  practices  for  safe  operations,  the  phase  of  flight, 
and  the  duration  of  any  irregularity.  For  example,  the  threshold  of  selected 
parameters  may  be  defined  for  various  altitudes,  e.g.,  1,000,  500,  250,  and 
100  feet,  during  landing  mode  events.  Typically,  40  to  80  events  are  defined 
and  analyzed  for  a  particular  aircraft.  For  example,  events  might  be  the 
ground  speed  during  taxi  or  the  descent  rate  during  approach.  The 
analysis  software  will  track  the  descent  over  time  to  calculate  a  rate  in 
terms  of  feet  per  minute.  Depending  on  the  aircraft’s  altitude,  a  descent 
rate  in  excess  of  specified  thresholds  will  trigger  an  exceedance.  Various 
categorization  schemes  are  used  to  classify  the  seriousness  of  the 
exceedance.  U.S.  airlines  use  two  or  three  categories  to  describe  the 
seriousness  of  exceedances,  ranging  from  minor  deviations  to  major 
deviations.  Exceedances  are  typically  specified  on  the  basis  of  a  strategy 
for  identifying  those  that  have  the  greatest  potential  for  safety  and 
performance  considerations.  Once  the  initial  exceedance  categories  and 
associated  parameters  have  been  defined  and  utilized,  they  are  subject  to 
an  ongoing  evaluation  and  refinement  process. 

The  ground  analysis  software  also  validates  the  qualify  and  integrity  of  the 
collected  data  and  filters  out  any  marginal  or  transitory  irregularities. 
Ground  analysis  systems  also  include  protective  mechanisms,  such  as  the 
de-identification  of  pilot  and  specific  flight  information  and  user  access 
privileges  based  on  assigned  passwords.  As  the  data  are  processed,  the 
flight  number  and  day  of  the  month  are  removed  and  saved  into  a  separate 
controlled  file.  This  step  “de-identifies”  the  foqa  data. 
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The  FOQA  monitoring  team  investigates  each  exceedance  to  determine 
what  occurred  and  the  severity  of  the  exceedance.  An  analyst  will  review 
the  parameter  values  surrounding  the  event  and  other  information  to 
determine  if  the  exceedance  was  valid  or  if  the  exceedance  was  based  on 
bad  data,  a  faulty  sensor,  or  some  other  invalidating  factor.  For  example, 
one  flight  had  excessive  rudder  input  on  landing  that  correctly  registered 
as  an  exceedance.  On  closer  examination,  it  was  determined  that  because 
the  aircraft  was  making  a  cross-wind  landing,  the  use  of  large  rudder  input 
was  justified.  In  this  case,  the  exceedance  was  deemed  invalid  and  was 
removed  from  the  exceedance  database. 

Depending  on  the  particular  circumstances  of  the  exceedance,  the  pilot 
association’s  representative  may  contact  the  flight  crew  to  gather  more 
information.  After  reviewing  the  situation  to  determine  the  exceedance’s 
cause,  the  foqa  monitoring  team  and  pilot  association’s  representative  will 
determine  any  necessary  corrective  action.  Corrective  action  can  range 
from  additional  flight  crew  training,  to  revisions  of  the  operating 
procedures,  to  redesigns  of  equipment. 

Trend  analysis.  On  aperiodic  basis,  airlines  aggregate  and  analyze 
exceedances  over  time — for  example,  the  number  of  unstabilized 
approaches  at  a  particular  airport  per  month,  over  the  last  12  months.  This 
type  of  analysis  provides  valuable  information  to  the  airline,  especially  in 
terms  of  whether  the  airline’s  performance  is  improving,  holding  steady,  or 
deteriorating.  This  look  at  aggregate  exceedances  over  time  provides 
airline  managers  with  a  new  perspective  on  potential  problems  that  would 
not  be  visible  otherwise.  On  the  basis  of  the  trend  analysis,  airline 
managers  can  take  corrective  action  to  reduce  or  eliminate  these 
exceedances  by  focusing  on  the  root  causes  and  making  or  recommending 
changes. 

Data  retention.  Detailed  foqa  data,  including  exceedances,  are  destroyed 
in  30  days  or  less  by  three  of  the  four  U.S.  airlines  with  foqa  programs. 
Trend  data,  however,  are  kept  indefinitely. 

Aircraft  equipping  decisions.  The  U.S.  airlines  with  active  foqa  programs 
have  each  equipped  a  portion  of  their  available  fleets  with  qars.  They 
began  their  programs  by  equipping  their  more  modem,  technically 
advanced  aircraft  with  qars — ^late-generation  aircraft  already  contain  the 
sensors  and  advanced  digital  systems  that  acquire  and  control  many  more 
flight  data  parameters  than  earlier-generation  aircraft.  Generally,  these 
airlines  do  not  plan  to  equip  any  of  their  older,  analog-based  aircraft,  such 
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as  Lockheed  L-1011,  McDonnell  Douglas  DC-9  and  DC- 10,  and  Boeing  727, 
737-100,  and  737-200,  with  qars  to  record  flight  data  because  these  aircraft 
would  be  expensive  to  retrofit  and  because  the  airlines  plan  to  retire  many 
of  them  in  the  near  future. 

Several  U.S.  airlines  plan  to  equip  all  new  aircraft  with  qars  or  other 
technology  to  capture  foqa  data.  Some  new  aircraft,  for  example,  are 
delivered  with  qars  as  standard  equipment.  Airlines  cited  several 
advantages  in  having  new  aircraft  delivered  with  factory-equipped  qars. 
One  advantage  is  that  aircraft  are  not  taken  out  of  service  to  be  retrofitted 
with  equipment.  Another  advantage  is  that  the  additional  cost  of  a  qar  can 
be  spread  over  the  finance  period  of  the  new  aircraft. 

Depending  on  the  specific  goals  of  a  foqa  program,  an  airline  may  wish  to 
equip  some  or  all  of  its  fleet  to  collect  flight  data.  If  a  program’s  goal  is  to 
identify  broad  trends  in  flight  operations  and  safety,  the  airline  may 
choose  to  equip  only  a  portion  of  its  fleet.  If  a  program’s  goal,  however,  is 
to  more  closely  monitor  the  flight  operations  and  performance  of 
individual  aircraft,  the  airline  may  want  to  equip  more  or  all  of  its  fleet. 
For  an  airline  that  begins  by  equipping  only  a  portion  of  its  fleet,  more 
aircraft  will  likely  be  added  to  the  program  so  that  these  data  can  be 
monitored  as  its  foqa  program  matures  and  efficiency  and  maintenance 
functions  are  added  to  the  program.  Some  U.S.  airlines,  for  example,  are 
planning  to  use  foqa  data  to  cut  aircraft  maintenance  costs  by  more 
closely  monitoring  engine  conditions  and  fuel  consumption. 


The  FOQA  Demonstration 
Project 


On  July  11, 1995,  faa  awarded  a  2-year  contract^^  to  execute  a  foqa 
demonstration  project,  referred  to  as  demoproj  by  faa,  to  Universal 
Technical  Resource  Services,  Inc.  (utrs),  of  Cherry  Hill,  New  Jersey.  The 
contract  stated  that 


“The  goal  of  demoproj  is  to  facilitate  the  start-up  of  the  foqa  initiative  and  to 
comprehensively  assess  the  cost-benefits  and  safety  enhancement  effectiveness  of  an 
implemented  foqa  program  in  which  airlines  voluntarily  employ  in-flight  recorded  data  to 
routinely  monitor  their  flight  operations.” 


UTRS  facilitated  the  establishment  of  collaborative  partnerships  between 
FAA,  UTRS,  and  interested  airlines.  Airhnes  may  participate  in  demoproj  at 
one  of  three  levels  within  the  project,  ranging  from  attending  meetings  and 
expressing  interest  to  a  full  partnership  with  faa.  Level  3  participation 


term  of  the  contract  was  later  revised  to  3  years. 
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refers  to  the  airlines  that  have  not  yet  established  an  official  foqa  program 
but  attend  meetings  to  learn  about  foqa.  At  Level  2,  the  airlines  already 
have  their  own  equipment  or  will  acquire  equipment  using  airline  funding, 
but  they  allow  utrs  to  monitor  and  document  their  program.  Level  1 
describes  a  full  partnership  in  which  equipment  and  software  are  provided 
through  DEMOPROJ.  Currently,  11  airlines  are  participating  in  dbmoproj.  The 
airlines  participating  at  Level  1  are  United,  US  Airways,  and  Continental. 
AD  other  participating  airlines  in  demoproj  are  at  Level  3:  Alaska,  America 
West,  Delta,  Northwest,  Trans  World,  Southwest,  Continental  Express,  and 
United  Parcel  Service. 

The  airline  participants  were  selected  on  the  basis  of  a  number  of 
characteristics,  including  financial  stability,  management  commitment, 
resource  commitment,  fleet  characteristics,  fleet  size,  aircraft  availability, 
and  an  approved  implementation  and  operation  plan.  AdditionaUy,  airlines 
are  required  to  sign  nondisclosure  and  cooperation  agreements  that  define 
the  treatment  of  confidential  and  proprietary  information,  enumerate  data 
access  control  and  security  provisions,  and  specify  the  responsibilities  and 
contributions  of  each  party.  Participating  airlines  also  had  to  secure 
agreements  with  their  pilot  associations  for  the  coUection  and  analysis  of 
flight  data  These  airlines  made  the  commitment  to  record  and  process 
FOQA  data  on  all  scheduled  flights  that  are  equipped  with  FAA-supplied 
equipment,  participate  in  periodic  project  reviews,  and  aUow  utrs  to 
interview  airline  personnel  during  the  project  to  document  procedures, 
problems,  issues,  and  solutions. 

UTRS  assisted  airlines  in  determining  the  equipment  best  suited  to  their 
needs,  acquiring  the  equipment,  and  delivering  it  for  instaDation  by  the 
airlines.  Hardware  and  software  were  selected  from  commercially 
avaDable,  off-the-shelf  sources.  As  part  of  this  effort,  the  contractor 
developed  an  Equipment  Overview  to  facilitate  the  airlines’  analysis  and 
selection  of  available  equipment. 

UTRS  also  monitors  and  documents  the  airlines’  foqa  demonstration 
programs’  policies,  procedures,  usage,  and  effectiveness.  The  contractor  is 
collecting  and  analyzing  information  on  how  each  airline  is  implementing 
FOQA,  including  data  processing  and  analysis;  the  retention  of  detaD  and 
trend  data^  the  selection  of  flight  data  parameters;  and  the  adjustment  of 
threshold  values,  system  effectiveness,  technical  problems,  and  resource 
information  for  establishing  and  maintaining  a  foqa  program.  These 
findings  are  integrated  and  disseminated  among  participants  throughout 
the  study,  utrs  is  also  collecting  information  about  the  projects’  costs  and 
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anticipated  benefits.  The  contractor  is  determining  how  each  airline 
transforms  foqa  data  into  information  and  how  this  information  is  used  in 
the  airline’s  decision-making,  utes  holds  periodic  meetings  for  all  partners 
to  promote  the  sharing  of  information  and  lessons  learned. 

UTRS,  with  airlines’  and  pilot  associations’  involvement,  is  developing  a 
FOQA  advisory  circular  to  provide  information  and  guidance  to  airlines  on 
how  to  design,  implement,  and  maintain  a  foqa  program.  This  document  is 
scheduled  to  be  issued  approximately  90  days  after  faa  issues  its  proposed 
rulemaking  on  enforcement  policy  in  connection  with  foqa.  utes  is  also 
developing  a  cost-benefit  analysis  that  will  provide  estimates  of  (1)  the 
costs  that  an  airline  would  incur  when  starting  and  maintaining  a  foqa 
program  and  (2)  potential  savings.  The  cost-benefit  study  is  scheduled  to 
be  completed  in  January  1998. 

UTES  will  issue  a  technical  report  and  a  set  of  foqa  guidelines  in  Jime  1998. 
The  technical  report  will  be  an  overall  description  of  the  technical  effort  to 
implement  foqa,  summarizing  the  airlines’  experiences  with  commercially 
available  equipment  and  systems.  The  foqa  guidelines  will  synthesize  the 
airlines’  experiences  in  implementing  foqa  with  a  view  toward  helping 
new  airlines  learn  firom  the  airlines  that  have  implemented  a  foqa 
program.  The  guidelines  will  include  information  on  (1)  designing  a  foqa 
program;  (2)  the  start-up  and  initial  operation  of  a  system;  (3)  the  use  of 
FOQA  for  trend  analysis,  knowledge  building,  and  decision-making;  and 
(4)  critical  success  factors  for  implementing  a  foqa  program. 

In  fiscal  years  1995  through  1997,  according  to  the  faa  foqa  program 
manager,  faa  allocated  $5.5  million  for  demoproj.  The  manager  stated  that, 
as  of  September  26, 1997,  demoproj  had  expended  $2.1  million,  including 
$1.1  million  for  the  purchase  of  hardware  and  software  for  the  three  Level 
1  airline  participants,  faa  plans  to  pursue  follow-on  development  focused 
on  the  acquisition  and  use  of  foqa  information  by  faa  for  safety  monitoring 
purposes. 


U.S.  Airlines  With  Active 
FOQA  Programs 


Currently,  four  U.S.  airlines  have  active  foqa  programs;  United  Airlines, 
US  Airways,  Continental  Airlines,  and  Alaska  Airlines.  These  airlines  have 
equipped  a  number  of  their  aircraft  with  qaes,  from  7  aircraft  at  Alaska 
Airlines  to  52  aircraft  at  United  Airlines.  The  number  of  parameters 
continuously  recorded  on  the  qaes  range  from  about  38  to  over  300, 
depending  on  the  airline  and  the  t3q)e  of  aircraft. 
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United  Airlines.  United  Airlines  has  the  largest  and  longest-running  foqa 
program  of  any  U.S.  airline,  begun  in  1995.  As  of  August  1997,  United  had 
52  aircraft  equipped  with  qars  and  had  collected  foqa  data  on  over  25,000 
flights.  The  aircraft  currently  equipped  include  Boeing  737-500s  and  777s 
and  Airbus  319s  and  320s.  United  plans  to  equip  over  120  aircraft  by  1999, 
including  all  new  aircraft  currently  on  order,  demopeoj  has  funded  the 
purchase  of  qars  to  equip  15  Boeing  737-500s  and  additional  data  analysis 
packages  and  computer  equipment  to  run  on  systems  that  United  had 
already  established.  The  remainder  of  the  hardware  and  software  was 
purchased  by  United,  which  has  been  tracking  and  correcting  exceedance 
events  for  more  than  a  year.  United  has  identified  and  taken  corrective 
action  to  reduce  the  incidence  of  a  number  of  safely-  and 
maintenance-related  exceedances. 

US  Airways.  US  Airways  has  23  QAR-equipped  aircraft.  Its  program,  begun 
in  September  1996,  has  collected  foqa  data  on  over  18,000  flights  to  date. 
Aircraft  equipped  include  Boeing  737-400s  and  767s.  US  Airways,  however, 
characterizes  its  program  as  being  in  the  data  collection  and 
trouble-shooting  phase  and  just  beginning  the  data  analysis  and  trending 
phase.  DEMOPEOJ  has  funded  the  purchase  of  qaes  to  equip  15  Boeing 
737400s  and  a  ground  analysis  system.  Six  additional  737400s  have  been 
equipped  with  qaes  paid  for  by  a  separate  faa  program,  the  Structural 
Loads  Program  (see  app.  HI).  In  addition  to  these  aircraft,  US  Airways  is  in 
the  process  of  purchasing  qaes  and  equipping  12  Boeing  767s.  Data  from 
all  QARS  are  being  accessed  by  both  programs,  demopeoj  has  also  funded  a 
trial  program  of  a  wireless  ground  datalink  system  with  five  specially 
equipped  Boeing  757s. 

Continental  Airlines.  Continental  has  equipped  15  Boeing  737-500s  with 
QARS.  In  addition.  Continental  plans  to  equip  with  qaes  all  new  aircraft  on 
order.  These  include  Boeing  737-500s,  -600s,  -700s,  and  -800s  as  well  as  a 
number  of  757s.  Begun  in  December  1996,  Continental’s  program  has 
analyzed  the  flight  data  from  over  11,000  flights  to  date.  According  to  the 
program  manager,  this  program  is  in  the  data  collection  phase  and  will 
soon  be  making  the  transition  to  the  data  analysis  and  trending  phase. 

Alaska  Airlines.  Alaska  Airlines  has  equipped  six  McDonnell  Douglas 
MD-80s  and  one  Boeing  737400  with  qars.  In  addition,  Alaska  has 
equipped  a  flight  simulator  with  equipment  to  record  hundreds  of  flight 
parameters.  Begun  in  July  1996,  the  program  has  analyzed  the  flight  data 
from  over  5,000  flights  to  date.  Still  in  the  early  stages  of  the  program. 
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Alaska  plans  to  “go  slow”  and  refine  its  program.  Alaska’s  foqa  manager 
said  that  the  airline  may  eventually  equip  every  aircraft  in  its  fleet. 

Unlike  United,  US  Airways,  and  Continental,  which  are  Level  1 
participants  in  demoproj,  Alaska  is  a  not  yet  a  fiiU  partner  in  demoproj 
because  it  has  only  recently  secured  the  required  agreement  with  its  pilot 
union  on  foqa.  The  airline,  however,  has  received  six  qars  and  a  ground 
analysis  system  from  faa’s  Structural  Loads  Program  (see  app.  III).  Alaska 
uses  the  equipment  and  analysis  system  for  both  the  Structural  Loads 
Program  and  foqa. 
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Adda  Airways 

Aeroflot 

Air  Afrique 

Air  France 

Air  Inter 

Air  Liberte 

Alaska  Airlines 

All  Nippon  Airways 

Asiana  Airlines 

Balkan  Airlines 

Britannia 

British  Airways 

British  Midland 

Cathay  Pacific  Airways 

China  Airlines 

China  Southern  Airlines 

China  South  West  Airlines 

Continental  Airlines 

Emirates 

Ethiopian  Airlines 

EVA  Airways 

Garuda  Indonesia 

GB  Airways 

Gulf  Air 

Japan  Airlines 

Royal  Dutch  Airlines  (klm) 

Kuwait  Airways 

Lufthansa 

Qantas  Airways 

Saudia  Arabian  Airlines 

Scandanavian  Airline  System  (SAS) 

Singapore  Airlines 

TAP  Air  Portugal 

Thai  Airways  International 

United  Airlines 

US  Airways 

Wideroe^® 


Source:  GAO  and  The  Flight  Data  Company,  Ltd. 


^^The  list  may  not  be  comprehensive. 
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Aviation  Performance  Measuring  System.  In  1993,  faa  contracted  with  the 
National  Aeronautics  and  Space  Administration  (nasa)  to  establish  and 
demonstrate  the  feasibility  of  developing  an  Aviation  Performance 
Measuring  System  (apms).  The  objective  of  the  apms  effort  is  to  develop 
tools  and  methodologies  to  allow  large  quantities  of  flight  data  to  be 
processed  in  a  highly  automated  manner  to  address  questions  relating  to 
operational  performance  and  safety,  apms  is  concerned  with  converting 
flight  data  into  useful  safety  information  in  support  of  the  national  air 
transport  system,  airlines,  and  air  crews.  Although  concerned  with  all 
aspects  of  flight  operations,  apms  primarily  will  develop  an  objective 
method  for  continuously  evaluating  air  crews’  technical  performance  in 
support  of  FOQA  and  the  Advanced  Qualification  Program  (discussed 
below). 

Current  foqa  programs  focus  on  exceedances;  apms,  however,  will  expand 
foqa’s  scope  by  utilizing  all  flight  data.  The  tools  will  facilitate  multiple 
functions,  including  the  acquisition  of  flight  data,  their  storage  in  a 
database  management  system,  the  study  of  statistical  characteristics  and 
trends,  the  development  of  “data  mining”  techniques,  and  better  methods 
of  visualizing  flight  data  apms  will  also  investigate  flight  animation 
capabilities  to  assist  flight  crews  in  replaying  and  understanding 
exceedances.  Finally,  apms  will  facilitate  the  sharing  of  data  among 
databases,  products,  and  interested  parties.  According  to  nasa  officials, 
one  of  the  most  important  components  to  be  developed  by  apms  is  a  risk 
assessment  tool  to  measure  how  much  risk  is  associated  with  certain 
activities,  for  example,  the  riskiness  of  flights  to/from  certain  airports. 

After  apms  began  in  1993,  the  project  documented  the  status  of  the 
technologies,  systems,  and  software  used  by  foreign  airlines  with  foqa 
programs.  According  to  the  nasa  project  manager,  the  project  has 
conducted  user  needs  studies  at  Alaska  Airlines,  United  Airlines,  and  US 
Airways  and  has  commitments  to  conduct  user  needs  studies  at  America 
West,  Trans  World  Airlines,  Comair,  and  United  Parcel  Service.  The  apms 
team  is  also  building  prototype  systems  at  several  airlines.  Alaska  Airlines 
is  now  in  its  third  prototype  apms  system.  The  project  was  scheduled  to 
begin  building  the  initial  prototype  system  at  United  Airlines  on 
November  1, 1997.  Eventually  the  developed  technology  will  be 
transferred  to  industry  so  that  a  relatively  low-cost  system  will  be 
commercially  available,  apms  management  hopes  to  initiate  the  transfer  of 
this  technology  to  commercial  vendors  in  12  to  18  months. 
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To  date,  nasa  has  received  $2.9  million  in  funding  from  faa  for  the 
development  of  apms.  nasa  contributed  $300,000  to  the  project  in  fiscal 
year  1997.  The  extent  of  future  nasa  and  faa  funding  for  further 
development  and  implementation  of  apms  has  not  yet  been  determined. 

Structural  Loads  Program.  As  part  of  faa’s  Aging  Aircraft  Research  and 
Development  Program,  the  Structural  Loads  Program  is  a  cooperative  faa 
and  NASA  effort  to  collect  information  about  the  external  loads  to  which 
airframe  components  are  subjected  during  flight.  The  collected  data  wiU 
be  used  to  develop  and  maintain  an  extensive  database  of  transport 
aircraft  usage  to  continuously  validate  and  update  flight  and  landing  load 
airworthiness  certification  standards  on  the  basis  of  actual  measured 
usage.  To  date,  the  Structural  Loads  Program  has  equipped  with  qars  six 
MD-80s  at  Alaska  Airlines  and  six  Boeing  737-400s  at  US  Airways.  Data 
collected  from  these  qars  are  also  being  made  available  for  foqa  analysis. 

Advanced  Qualification  Program,  faa’s  Advanced  Qualification  Program 
(AQP)  is  an  alternate  method  of  qualifying,  training,  certifying,  and  ensuring 
the  competency  of  flight  crew  members  and  other  operations  personnel 
subject  to  the  training  and  evaluation  requirements  of  Federal  Aviation 
Regulation  (far)  parts  121  and  135.  aqp’s  intent  is  to  achieve  the  highest 
possible  standards  of  individual  and  crew  performance  without  imdue 
increases  in  training  costs,  foqa  and  apms  will  be  used  to  continuously 
evaluate  air  crews’  technical  skills  and  airlines’  procedures  and  training  in 
support  of  AQP.  For  example,  foqa  data  could  be  used  to  identify  problems 
occurring  during  recurrent  flight  simulator  training  and  to  highlight 
training  areas  for  increased  emphasis. 

Global  Analysis  and  Information  Network.  The  Global  Analysis  and 
Information  Network  (gain)  is  a  concept  being  actively  explored  by  the 
aviation  community  to  facilitate  the  analysis,  sharing,  and  dissemination  of 
aviation  safety  information  with  a  goal  of  achieving  zero  accidents,  gain 
would  have  many  information  sources — ^poqa  information  would  be  one  of 
the  most  important.  Proposed  by  faa  in  May  1996,  gain  will  function  as  a 
“significantly  improved  operational  early  warning  capability  that  is 
sensitive  enough  to  detect  and  alert  the  aviation  community  to  existing 
and  emerging  problems.”  Information  will  be  shared  among  airlines  and 
manufacturers  and  at  the  different  functional  levels  within  organizations. 
Although  GAIN  is  still  in  the  conceptual  phase,  the  aviation  community  and 
FAA  are  working  to  address  the  needs  and  concerns  of  prospective 
members  as  well  as  explore  potential  designs  for  a  protofype  system. 
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FAA  has  implemented  a  number  of  voluntary  programs  involving  the 
self-reporting  of  safety-related  information  to  enhance  aviation  safety. 
Although  these  programs  involve  the  reporting  of  information  by  people 
instead  of  by  automated  systems,  they  are  similar  to  foqa  in  that  they 
involve  voluntary  efforts  to  identify  and  correct  potential  safety  problems. 
We  have  highlighted  three  such  programs. 

Aviation  Safety  Reporting  Program.  Established  by  faa  in  1975  and 
administered  by  nasa,  the  Aviation  Safety  Reporting  Program  (asrp) 
promotes  the  voluntary  reporting  of  problems  into  the  Aviation  Safety 
Reporting  System  (asrs).  Under  14  C.F.R.  91.25,  faa  will  not  use  reports 
submitted  under  the  program  in  any  enforcement  action  (except  accidents 
or  criminal  offenses).  Under  faa’s  policy,  although  a  finding  of  violation 
may  be  made,  no  sanction  will  be  imposed  if  (1)  the  violation  was 
inadvertent  and  not  deliberate,  (2)  the  violation  did  not  involve  a  criminal 
offense  or  an  accident  or  an  action  that  discloses  a  lack  of  qualification  or 
competency,  (3)  the  person  filing  the  report  has  not  been  found  in  any 
prior  FAA  enforcement  action  to  have  committed  a  violation  of  federal 
aviation  regulations  or  law  within  a  period  of  5  years  prior  to  the 
occurrence,  and  (4)  the  report  was  filed  within  10  days  after  the  violation. 
AC  00-46D  (Feb.  26, 1997),  “Aviation  Safety  Reporting  Program.”^'’’ 

Air  Carrier  Volimtary  Disclosure  Reporting  Procedures.  Irutiated  by  faa  in 
1990  for  air  carriers,^®  the  Voluntary  Disclosure  Procedures  encourage 
airlines  to  promptly  disclose  to  faa  any  instances  of  noncompliance  with 
the  requirements  for  maintenance,  flight  operations,  and  security,  faa 
initiated  a  policy  of  forgoing  civil  penalty  actions  if  five  conditions  are  met: 
(1)  the  certificate  holder  immediately  notifies  faa  of  the  apparent  violation 
after  detecting  it  and  before  the  agency  learns  of  it;  (2)  the  apparent 
violation  is  inadvertent;  (3)  the  apparent  violation  does  not  indicate  a  lack, 
or  reasonable  question,  of  the  basic  qualification  of  the  certificate  holder; 
(4)  immediate  action  must  have  been  taken,  or  begun,  upon  discovery  to 
terminate  the  conduct  that  resulted  in  the  apparent  violation;  and  (5)  the 
certificate  holder  must  develop  and  implement  a  comprehensive  solution 
satisfactory  to  the  faa.  AC  120-56  (Jan.  23, 1992),  “Air  Carrier  Voluntary 
Disclosure  Reporting  Procedures.” 


^'^Provisions  concerning  air  traffic  controllers  involved  in  incidents  reported  under  ASRS  are  addressed 
in  FAA  Order  7210.3.G,  Facility  Operations  and  Administration. 

^^According  to  an  assistant  chief  counsel  at  FAA»  the  procedures  have  since  been  expanded  to  include 
production  approval  holders  and  repair  stations. 
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Aviation  Safety  Action  Programs,  faa  has  established  several 
demonstration  Aviation  Safety  Action  Programs  (asap),  including  the 
USAir  Altitude  A'wareness  Program,  the  Alaska  Airlines  Altitude 
Awareness  Program,  and  the  American  Airlines  Safety  Action  Program.^® 
These  programs  established  incentives  to  encourage  the  employees  of  the 
air  carriers  that  are  participating  in  the  programs  to  disclose  information 
and  identify  possible  violations  of  the  Federal  Aviation  Regulations 
without  fear  of  punitive  legal  enforcement  sanctions,  faa  has  recently 
expanded  the  use  of  asap  through  the  implementation  of  a  2-year 
demonstration  program.  Under  this  program,  apparent  violations  will 
normally  be  addressed  with  administrative  action  if  the  apparent 
violations  do  not  involve  (1)  deliberate  misconduct,  (2)  a  substantial 
disregard  for  safety  and  security,  (3)  criminal  conduct,  or  (4)  conduct  that 
demonstrates  or  raises  a  lack  of  qualifications.  For  apparent  violations  not 
excluded  under  an  asap,  neither  administrative  action  nor  punitive  legal 
enforcement  actions  will  be  taken  against  an  individual  unless  there  is 
sufficient  evidence  of  the  violation  other  than  the  individual’s 
safety-related  report.  AC  120-66  (Jan.  8, 1997),  “Aviation  Safety  Action 
Programs  (asap).” 


2®The  Altitude  Awareness  programs  at  USAir  and  Alaska  Airlines  were  joint  programs  with  the  Air  Line 
Pilots  Association  and  FAA  to  eliminate  altitude  deviations.  USAir^s  program,  in  operation  from 
October  1990  through  February  1992,  and  Alaska  Airlines'  program,  in  operation  from  August  1994 
throu^  February  1995,  enco\u*aged  fli^t  crews  to  report  altitude  problems  so  that  corrective  action 
could  be  taken.  The  American  Airlines  Safety  Action  Program  is  a  joint  program  with  the  Allied  Pilots 
Association  (American's  pilot  union)  and  FAA.  Begun  in  June  1994,  the  program  encourages  pilots  to 
report  all  types  of  potential  safety  problems. 
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Airline  officials  and  pilot  unions’  representatives  are  concerned  about  the 
use  of  discovery  in  civil  litigation  to  reveal  voluntarily  collected  safety 
information.  In  two  recent  cases,  the  courts  have  sought  to  find  a  balance 
between  the  airlines’  desire  to  protect  such  information  and  the  plaintiffs’ 
right  to  a  fair  trial.  In  one  case,  the  documents  were  required  to  be 
produced,  but  under  a  protective  order.  In  the  other  case,  the  court 
recognized  a  new  qualified  privilege. 

In  1995,  the  United  States  District  Court,  District  of  South  Carolina, 
Columbia  Division,  rejected  USAir,  Inc.’s*’  argument  that  certain  safety 
data  were  protected  under  the  self-critical  evaluation  privilege.  Court 
Order  of  Oct.  26, 1995,  In  re  Air  Crash  at  Charlotte,  North  Carolina,  on 
July  2, 1994,  MDL  Docket  No.  1041  (D.S.C.  1995).  This  privilege,  when 
recognized,  protects  documents  that  reflect  self-analysis. 

The  district  court  noted  that  the  self-critical  evaluation  privilege  is  a 
privilege  of  recent  origin  and  one  that  is  narrowly  applied  even  in  those 
jurisdictions  where  it  is  recognized.  The  court  described  the  privilege  by 
citing  to  Dowling  v.  American  Hawaii  Cruises,  Inc.,  971  F.2d  423,  425-426 
(9th  Cir.  1992),  which  explained: 

“(0]ther  courts  have  generally  required  that  the  parly  asserting  the  privilege  demonstrate 
that  the  material  to  be  protected  satisfies  at  least  three  criteria:  first,  the  information  must 
result  from  a  critical  self-analysis  undertaken  by  the  party  seeking  protection;  second,  the 
public  must  have  a  strong  interest  in  preserving  the  free  flow  of  the  type  of  information 
sought;  finally,  the  information  must  be  of  the  type  whose  flow  would  be  curtailed  if 
discovery  were  allowed. . .  .To  these  requirements  should  be  added  the  general  proviso  that 
no  document  will  be  accorded  a  privilege  imless  it  was  prepared  with  the  expectation  that 
it  would  be  kept  confidential,  and  has  in  fact  been  kept  confidential.” 


The  court  found  that  the  safety  documents  did  not  meet  the  criteria  for  the 
privilege.  According  to  the  court,  the  most  significant  stumbling  block  for 
the  airline  was  meeting  the  third  criterion — ^that  the  flow  of  the 
information  would  be  curtailed  if  discovery  was  allowed.  Specifically,  the 
court  found  that  the  airline  industry  is  highly  competitive  and  tightly 
regulated  and  that  airlines  have  a  keen  interest  in  advancing  and 
promoting  safety  as  well  as  services.  Thus,  the  court  reasoned  that  the 
airlines  were  likely  to  conduct  internal  audits.  The  coiut  reasoned  that 
while  the  disclosure  of  such  audits  to  competitors  would  deter  their  use  in 
the  future,  disclosure  for  limited  use  in  litigation  is  imlikely  to  have  such 
an  impact. 


^^SAir  changed  its  name  to  US  Airways  on  Feb.  27, 1997. 
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Subsequently,  the  court  limited  the  possible  uses  of  the  documents  it 
ordered  to  be  produced.  Specifically,  the  court  ordered: 


. .  plaintiff  and  their  counsel  shall  be  prohibited  from  disclosing,  disseminating  or 
communicating  in  any  manner  to  any  person  or  entity  not  involved  in  this  litigation  any 
portion  of  the  information  contained  in  those  documents. . . .  Plaintiff  and  their  counsel 
shall  be  further  precluded  from  utilizing  these  documents  or  the  information  contained  m 
them  for  any  purpose  other  than  for  this  multidistrict  litigation. 


“In  furtherance  of  this  order,  plaintiffs’  counsel  shall  insure  that  each  person  who  is  to  be 
given  access  to  the  referenced  documents,  including  plaintiff  and  their  attorneys,  shall  first 
sign  a  document  acknowledging  that  they  are  aware  of  and  will  comply  with  this  order. 
Plaintiffs’  counsel  shall  maintain  a  list  of  those  persons  which  shall  be  provided  to  USAir’s 
attorney  upon  request,  subject  to  protection  upon  application  to  this  court  for  good  cause 
shown.”  Court  Order  of  Nov.  14, 1995,  In  Re:  Air  Crash  at  Charlotte,  North  Carolina,  on 
July  2, 1994,  MDL  Docket  No.  1041  (D.S.C.  1995). 

In  October  1996,  the  Supreme  Court  let  stand  the  district  court  order 
rejecting  the  airline’s  assertion  of  a  self-critical  evaluation  privilege.  65 
U.S.L.W.  3221  (Oct.  8,  1996). 

Recently,  in  another  case  involving  documents  prepared  by  American 
Airhnes’  employees  collected  under  the  American  Airlines  Safety  Action 
Program,  the  United  States  District  Coiut,  Southern  District  of  Florida,  on 
a  motion  for  reconsideration,  also  rejected  the  airline’s  self-critical 
analysis  privilege  claim.  However,  in  this  case  the  court  recognized  a  new 
qualified  privilege  to  protect  these  documents.  In  re  Air  Crash  Near  Cali, 
Colombia,  on  Dec.  20, 1995,  959  F.  Supp.  1529  (S.D.  Fla  1997). 

With  respect  to  the  self-critical  analysis  privilege,  the  court  stated  that  “the 
touchstone  of  a  self-critical  analysis  is  that  it  is  an  ’in  house’  review 
imdertaken  primarily,  if  not  exclusively,  for  the  purpose  of  internal  quality 
control.”  In  this  case,  the  court  rejected  the  jqiplication  of  the  privilege, 
finding  the  following: 

“Even  assuming  that  the  materials  prepared  by  American’s  pilots  in  coiyimction  with  the 
ASAP  program  may  be  of  a  type  whose  creation  might  be  curtailed  if  discovery  is  allowed, 
these  materials  were  prepared  for  dissemination  to  representatives  of  entities  imaffiliated 
with  American  (a  federal  regulatory  agency  and  a  union).” 

The  court,  however,  recognized  a  new,  qualified  common  law  privilege  for 
the  ASAP  materials.  In  recognizing  anew  privilege,  the  court  considered  the 
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principles  for  evaluating  claims  of  federal  common  law  privileges  recently 

articulated  in  the  Supreme  Court  case,  Jaffee  v.  Redmond, _ U.S. _ ,  116 

S.  Ct.  1923  (1996):  (1)  the  “private  interest”  involved — ^in  other  words 
whether  the  dissemination  of  the  information  would  chill  the  frank  and 
complete  disclosure  of  fact;  (2)  the  “public  interests”  furthered  by  the 
proposed  privilege;  (3)  the  “likely  evidentiary  benefit  that  would  result 
from  the  denial  of  the  privilege;”  and  (4)  the  extent  to  which  the  privilege 
has  been  recognized  by  state  courts  and  legislatures.®^ 

The  court  found  that  American  had  met  its  burden  of  proving  that  a 
qualified  “asap  privilege”  is  appropriate.  Specifically,  the  court  stated  as 
follows: 

“The  ASAP  materials  in  dispute  . . .  were  prepared  voluntarily,  in  confidence  and  for  use  in  a 
discrete,  limited  context  in  cooperation  with  the  faa  and  the  pilot’s  union.  There  is  a 
genuine  risk  of  meaningful  and  irreparable  chill  from  the  compelled  disclosure  of  asap 
materials  in  connection  with  the  pending  litigation.” 

The  court  specified  that  the  privilege  should  be  qualified.  Accordingly,  the 
plaintiffs  could  overcome  the  privilege  with  a  persuasive  showing  of  need 
and  hardship.  The  plaintiffe  did  not  make  such  a  showing  in  the  case. 


®^The  court  recognized  a  psychotherapist-patient  privilege  in  this  case. 
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